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Abstract 

In a standard NP-complete optimization problem we introduce an 
interpolating algorithm between the quick decrease along the gradi- 
ent (greedy dynamics) and a slow decrease close to the level curves 
(reluctant dynamics) . We find that for a fixed elapsed computer time 
the best performance of the optimization is reached at a special value 
of the interpolation parameter, considerably improving the results of 
the pure cases greedy and reluctant. 



1 Introduction and results 

Combinatorial optimization stands as one of the most fruitful fields in the 
intersection of applied and pure mathematics: it connects the theory of 
the computational hardness to the techniques widely used in the search 
for global minima for complex functionals, i.e. functions with many lo- 
cal minima. Over the years there have been many strategies proposed to 
solve efficiently hard computational problems PQ 12 El- Among them the 
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statistical mechanics approach jH |5] has opened new interesting perspec- 
tives. In this paper we study the interpolation between two algorithms, 
the greedy and the reluctant. The first is the standard decrease along the 
deepest descent direction, while the second is the closest decrease to the 
level lines. In previous works Ej we have studied and compared the two 
algorithms focusing on relaxation time and minimum reached level. We 
observed, moreover, how a simple convex interpolation between them could 
improve the performances of both of them in large size regime. In this work 
we push further such analysis introducing a smooth interpolation between 
greedy and reluctant depending on a parameter A: small A plays the role 
of the greedy algorithm, while large A that of reluctant. This allows us a 
better tuning among the two and especially a parameter optimization. The 
newly introduced algorithm is tested against a model which has become the 
standard of NP complete problems: the Sherrington Kirkpatrick model of 
the mean field spin glass. Our results confirm and extend those in [SJ [7] : 
we find that the relaxation time grows linearly when the algorithm is close 
to the greedy regime and quadratically when it is close to the reluctant one 
with a progressive condensation for large values of A. The dynamics is then 
tested in the search for low energy configurations for fixed values of initial 
conditions, where the reluctant dynamics works substantially better than 
any other. The main result of this work is then the minimization at fixed 
elapsed computer time. In this case, in fact, we find that in the small size 
regime (compared to total search time) the greedy component performs bet- 
ter than any other due the short relaxation time and the fact that, basically, 
the dynamics is able to find the ground state or, at least, to get very close to 
it. Moreover and more interestingly, we find that increasing the system size 
does not lead to a uniform deteriorating of the greedy performance toward 
an improvement of the reluctant one. We find, in fact, an optimal value 
of A ~ 10, for which the lowest energies are reached against a rather poor 
performance of the greedy (A ~ 1) and reluctant algorithm (A ~ 100). This 
optimal value appears to be independent of the size. 

2 The model and the algorithm 

Let us consider the Sherrington-Kirkpatrick model jS] defined by the Hamil- 
tonian 
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where <7j = ±1 for i = 1, . . . , N are spin variables which interact through 
an N x N symmetric matrix with independent, identically distributed 
gaussian random couplings (Jij = Jji, Ju = 0) with zero mean and variance 
1/N. We focus our attention on a stochastic energy-decreasing dynamics 
that, starting from any initial spin configuration at time t = (which we 
choose at random with uniform distribution), ends up on a local energy 
minimum. The evolution rule is: 

1. Let a{t) = (o"i(t), . . . , ^(t)) be the spin configuration at time t. 

2. Calculate the spectrum of energy change obtained by flipping the spin 
in position i, for i — 1, . . . , N: 

AE i = <r i (t)^2j ij a j (t). (2) 

If AEi > 0, Vi = 1, . . . , N, then the algorithm stops (cr(t) is a local 
minimum) . 

3. Generate a random number D with probability density 

, f \e Xx if x < . . ... 

/(*) = ( o iix ~ , A>0. (3) 

4. Select the site i* associated with the closest energy change to the value 
D, i.e.: 




i e {!,..., N} : AE^ = . min {\AEi — D\ : AEi < 0} >. (4) 



5. Flip the spin on site i*: 

^ (t+1) = (^(t) if < ^ i*. (5) 

This algorithm generates a dynamics that, following a 1-spin flip decreasing 
energy trajectory, arrives at a 1-spin flip stable configuration, that is a 
configuration whose energy cannot be decreased by a single spin-flip. The 
speed of convergence to local energy minima is tuned by A, the control 
parameter in the probability distribution function for the move acceptance. 
Of course, the larger is A, the bigger it is the probability of doing small 
energy-decreasing steps, so that the trajectory will follow an evolution path 
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close to level curves (reluctant). On the other hand, small values of A enrich 
the probability of large negative energy steps, which will quickly drive the 
dynamic to the end-point (greedy). In the next Section, by varying the 
control parameter A, we study the efficiency of the algorithm by measuring 
the average time to reach a metastable configuration and the lowest energy 
value found as the system size is increased. 



We performed a set of experiments for different values of N, starting from 
N initial conditions (for a system of size N) and averaging the data on 
ureal = 1000 disorder realizations. We probed basically two quantities to 
measure the performance of the algorithm: 

• the average time (i.e. the number of spin flips) to reach a minimum 
energy level 



with M = N ■ ureal and U, % = 1, . . . , M the time for each initial 
condition 

• the lowest energy found (averaged over disorder) 



where min CT Hn( J, a) is the minimum value of the energy of the me- 
tastable states attained starting from N initial conditions. 

In Fig.l we represent r as a function of N in the range [25, 300] for six dis- 
tinct values of A (A = 1, 10, 25, 45, 70, 100), together with the best numerical 
fits. Because of high computational costs (which increase with A), the case 
N = 300 is studied in details only for A = 1 and A = 10. On the other 
hand, the average time has good self-averaging properties so that, in order 
to have the trend for "large" A we focused on the case N = 300, A = 100 
with ureal = 140 disorder realizations instead of ureal = 1000. From Fig.l 
we observe the progressive increase of the slope in log-log scale from an 
almost linear law in N for A = 1 (o) to an almost quadratic one for A = 100 
(*). Thus, this algorithm behaves as a "smooth interpolation" between the 
two deterministic dynamics: "greedy", that we obtain for A = 1, and "re- 
luctant", here represented by A = 100. More in detail, the numerical fits of 
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Figure 1: Average time r to reach a metastable configuration as a function 
of iV for A = f (o), A = 10 (+), A = 25 (□), A = 45 (x), A = 70 (A), 
A = 100 (*). 



Fig.l are: r x=1 (N) ~ iV 1 - 027 (o), r x=w (N) ~ TV 1 - 263 (+), r A=25 (iV) ~ TV 1 - 600 
(□), T A=45 (iV) ~ jv 1 - 796 (x), r x=70 (N) ~ AT 1 - 911 (A), r A=100 (iV) ~ jv 1 - 932 
(*). The fits are quite good for all the cases but A = 10 and A = 25. 
In these cases the quality of the fit is enhanced excluding the data corre- 
sponding to N = 25 and N = 40. So we obtain r A= i (iV) ~ iV 1 - 184 and 

TA=2 5 (A0 ~ iV 1 " 488 . 
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Figure 2: Lowest energy value Hn as a function of N obtained using a 
protocol with N initial conditions for 1000 disorder realizations for A = 1 
(o), A = 10 (+), A = 25 (□), A = 45 (x), A = 70 (A), A = 100 (*). 



Next, we measured the lowest energy found by the algorithm. In Fig. 2 
we represent Hn as a function of N for different values of A and for a fixed 
number of N initial conditions. While, for small N, the ground state is be- 
lieved to be closely approximated for all values of A (in fact, varying A, the 
lowest energy H N undergoes a relative change of 2.5 • 10 -4 for iV = 25 and 
of 8.8 • 10~ 4 for iV = 40 and 50), the best results for large N (> 50) are ob- 
tained for A = 100 which corresponds to deterministic reluctant dynamics. 
Therefore, this confirms that, for a fixed number of initial spin configu- 
rations, the algorithm that makes moves corresponding to the "smallest" 
possible energy decrease is the most efficient in reaching low-energy states. 
In other words, the slower . . . the better! However, this is reflected in an 
increasing cost for the computational time. 

Indeed, when the analysis is focused on the performances for a fixed 
elapsed time, the situation changes drastically. In Fig. 3 we compare the 
minimum energy values H N , obtained considering different system sizes 
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and, for each of them, five different parameter values (A = 1, A = 10, A = 
25, A = 45, A = 100) for an elapsed time of 50 h of CPU on a IBM SP4. 
The system size N = 350 is studied only for A = 10, A = 25 and A = 45. 
Each run (i.e. for fixed iV and A) consists of 1000 disorder realizations, 
with the same CPU time length (3 min.) assigned to each sample. For 
N < 100 we believe to find the ground state of the system, since varying 
A the values of coincide, within our numerical accuracy (10~ 10 ). The 
best result is obtained for the case A = 10, which seems to be the best 
compromise to obtain a dynamical trajectory that is able to arrive deep 
enough with respect to energy levels but without wasting all the time in the 
search for the slower of possible path. We note that this finding is in good 
agreement with the result of previous analysis [6], where a convex linear 
combination of reluctant (with probability P) and greedy (with probability 
1 — P) dynamics was considered. The optimal value A ~ 10 is the one for 
which the relaxation time r ~ N a grows with a scaling exponent a which is 
the closest, among the others, to the value a = 1.26 of the optimal convex 
combination with P = 0.1. 

Improvements of the greedy and reluctant algorithms is presently under 
study jH], by permitting also increase in energy with exponential decrease 
in time, in the very same spirit of the well-known Simulated Annealing 
strategies. 
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